Why does Qdrant merge and re-rank results from multiple shards rather than simply concatenating each shard's top-k?

Because each shard holds only a subset of the collection's points, a shard's top-k is the top-k within that subset, not within the whole collection. If you simply concatenated the per-shard top-k lists, you would get a list of k times shard_count candidates, and the true global nearest neighbors could be ranked below candidates that happened to be in the same shard as each other. The correct global top-k requires two things: each shard must return more than k candidates (at least k plus any offset, and often more to account for the fact that the global ranking can pull from any shard), and the coordinator must merge the candidate sets by score and take the global top-k. This is the standard distributed top-k merge, and it is the only way to get the same result you would get from a single-shard search.

The mechanism is straightforward but easy to get wrong. The coordinator sends the query to every relevant shard with a per-shard limit that is at least the global limit plus the offset. If the query has offset=100 and limit=10, each shard must return 110 results, because the global 11th through 110th results could all come from a single shard. If the query involves a fusion stage (for example, Reciprocal Rank Fusion of a dense and sparse retriever), the merge is not a score sort but a rank-based fusion: the coordinator fuses the per-shard lists before taking the global top-k. This is why the fusion semantics matter: merging by score and merging by rank produce different results, and the coordinator has to apply the same fusion that a single-shard query would. For multi-stage queries expressed with prefetch, each stage's merge happens at the coordinator, and the candidate set passed to the next stage is the merged result, not the concatenation of per-shard lists.

The trade-off is result correctness against coordinator cost. Requesting more than k from each shard is necessary for correctness but increases the data transferred to the coordinator and the cost of the merge. The coordinator's work is proportional to shard_count times per_shard_limit, so at high shard counts and large limits the merge itself becomes a bottleneck. The common mistake is thinking that concatenating per-shard top-k is a reasonable approximation. It is not - it can systematically favor shards that happen to contain many similar points, and the error is not bounded. The second mistake is forgetting the offset. A query with offset=1000 and limit=10 that requests only 10 per shard will return the wrong results, because the global top-1010 almost certainly includes more than 10 results from at least one shard. The third mistake is applying a score-based merge to a hybrid query that used rank-based fusion on the shards; the merge must use the same fusion function. Version note: the prefetch/fusion API and the exact merge semantics for multi-stage queries are recent additions and have changed across releases; the coordinator's behavior under fusion is version-specific.

Version-dependent: the prefetch and fusion API, the supported fusion modes (RRF, DBSF), and the exact merge semantics at each stage have evolved across releases. In older versions, multi-stage retrieval was orchestrated in the application layer, which meant the merge happened in the client and the per-shard limit was your responsibility. In newer versions, the coordinator handles the nesting, and the per-shard limit is derived from the stage's limit. If you are debugging a result discrepancy between a single-shard and multi-shard deployment, check the version's merge behavior first, because it is the most likely source of a subtle difference.